Skip to content

Label each pull request once, the first time review-prs reads it - #206

Merged
bbondy merged 2 commits into
masterfrom
review-prs-label-prs
Oct 31, 2026
Merged

bbondy merged 2 commits into
masterfrom
review-prs-label-prs

Conversation

@netzenbot

@netzenbot netzenbot commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

User impact: a pull request on brave/brave-core or brave/bravebot gets pr/ labels such as pr/risk/network-calls or pr/tests-only from the first /review-prs run that reads it, once the labels exist in the repository.

The problem

Nothing on a pull request says which review it needs. Whether it needs a human, a security reviewer, or only an AI review is decided by reading it, every time. The brave-core discussion of the review process proposes labels for risk (funds, security, privacy promises, network calls, dependencies) and for changes that need little review (tests only, chromium_src overrides).

Reproduce

  1. Open a pull request on brave/brave-core that adds a file under chromium_src/ and changes a *_unittest.cc file.
  2. Run /review-prs #<number> auto, or wait for the scheduled run.
  3. Read the labels on the pull request.

None of the labels below is on it. Expected: pr/chromium-overrides, plus whichever of pr/risk/funds, pr/risk/security, pr/risk/privacy-promises-affected and pr/risk/network-calls the diff shows, each added once.

I have not run these steps against a live pull request. tests/test_pr_labels.py and the new cases in tests/test_review_prs.py cover the decisions with gh stubbed, and fail on the parent commit, where scripts/lib/pr_labels.py does not exist.

The fix

review.labelPrs in a project's profile lists the labels. Paths decide some of them with no model (pr/tests-only, pr/docs-only, pr/chromium-overrides, pr/risk/dependencies). When the profile also lists risk labels no path can decide, prepare-review.py writes one more prompt for the PR and one subagent chooses from that list. collect-results.py adds the labels after posting and records the PR in _labelled in the review cache, so a later push, a re-review or a label a person removed leaves it alone, and the model is asked once per PR. The subagent can only pick from the profile's list, labels are never removed, and a label that would lower scrutiny is not judged by a model.

Nothing is applied until every label the profile names exists in the repository, because the reviewer account has only triage on brave-core. A person creates them first; the commands are in the "Pull request labels" section of projects/brave-core/docs/labels.md and projects/bravebot/docs/labels.md. Every label starts with pr/, so none is one the repository already had.

Test plan

  • ruff check . and ruff format --check . — passed
  • pytest tests/ — 1582 passed, 4 failed; the same 4 fail on origin/master (1423e1d), and Make the lock tests and one review test pass on macOS #205 fixes them
  • ten deliberate breaks of the "once", "only from the list", dry-run, docs-only and prefix behaviours — each made a test fail
  • make check-reviewdog — no findings
  • gh label list against brave/brave-core and brave/bravebot — none of the 13 labels exists yet, so nothing would be applied
  • The labels are created in both repositories
  • A /review-prs run against a real pull request adds them
  • CI passes cleanly

review-prs now adds labels that say which review a PR needs. Paths decide
tests-only, chromium-overrides, docs-only and risk/dependencies with no model.
One Sonnet subagent per PR chooses among the profile's risk/* labels by reading
the diff: funds, security, privacy-promises-affected and network-calls for
brave-core, guarantee and network-calls for bravebot. Each project names its
labels in review.labelPrs of its profile.

A PR is labelled once. The cache records it in _labelled, so a later push, a
re-review or a label a person removed is left alone, and the model is never
asked about the same PR twice. The subagent chooses from the profile's list
only, a label that would lower scrutiny is never judged by a model, and labels
are only added.

Nothing is applied until every label exists in the PR repository, because the
reviewer account has triage on brave-core and cannot create them. Until then
no prompt is written, no model runs and no PR is recorded. The commands to
create the labels are in each project's docs/labels.md.
Every label review-prs puts on a pull request now starts with pr/, for example
pr/risk/dependencies, so none of them is a label the repository already had.
brave-core's tests, dependencies and security labels are left as they are. A
test holds the shipped profiles to the prefix.

brave-core gets pr/docs-only, which applies when every changed file is a .md
file. It does not apply to AGENTS.md, SECURITY.md, .claude/, agents/,
docs/best_practices.md, docs/best-practices/, or the documentation of vendored
code under third_party/ and tools/crates/vendor/, because those steer an agent
or the reviewer, state the security policy, or belong to a dependency. The
bravebot pr/docs-only now also leaves out the two files its pull requests are
reviewed against.
@bbondy
bbondy merged commit 15d6446 into master Oct 31, 2026
3 checks passed
@bbondy
bbondy deleted the review-prs-label-prs branch October 31, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants